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Abstract —In this paper, we propose a novel formulation 
to understand the tradeoff between binning utility and meta 
information leakage when we face the problems of partitioning 
random items. As an example, such problems could emerge 
when online users attempt to protect their browsing behavior 
patterns to certain extent by resorting to multiple proxy 
websites. Under the framework, we formulate a constrained 
optimization problem where the goal is to maximize binning 
utility while restraining a certain level of information leakage 
by properly dividing a set of M random items into N bins. By 
doing so, we formulate a new multi-agent multi-variate opti¬ 
mization problem which is NP-complicated. We then utilize the 
submodular nature of the problem to find sufficient conditions 
to (1) secure the existence of a solution to our problem; (2) lower 
the complexity of the problem at the cost of accuracy. To do so, 
we exploit the dual nature of set functions in multi-agent multi¬ 
variate problems, a novel addition to the field. After proving 
sufficient conditions to secure the existence of a solution in 
more general cases, we offer algorithms and complexity orders 
to solve a simplified version of the problem where N = 2 which 
helps signify the use of submodular properties. 

Index Terms —partition, information leakage, privacy, mutual 
information, submodularity 

I. Introduction 

Today internet has become so intertwined with our every¬ 
day activities that it is impossible to imagine living without 
it. However, this dependency could be subject to exploitation 
by the eavesdroppers. Assuming every browse as a query, it 
could be argued that the search history of a user contains 
a series of queries specific to him which could point to his 
specific likes and dislikes. By following every user’s history 
of browses vital information specific to each user could be 
developed. It then becomes important that each user attempt 
to add some level of protection to their browses to hide 
necessary -or rather enough information about themselves. 

One method is the use of proxy websites. Such websites 
offer the user a URL box where he can input any website he 
wishes to visit. The only difference is that in such websites, 
the address input is encoded into a series of characters which 
appear at the end of the URL of the original proxy website. 
These characters change through time by seconds meaning 
if the service provider records the URL opened through 
the proxy website and decides to open it to access specific 
content by inputting the URL, he will not go to the encoded 
web page. Also, such websites slow the connection. However, 
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through this method, the user has the option of choosing 
multiple proxy websites and thus presumably cutting down 
on the utility loss. Thus if a utility function based upon 
connection speed -bandwidth- for the user is calculable, 
a privacy constrained problem between the user and an 
eavesdropper (for example a service provider) could be 
defined. The solution to such a problem could offer insights 
in regards to the tradeoff between proxy allocation utility 
and meta information leakage when we face the problems 
of partitioning a set of random items (i.e. websites that a 
user has chosen to visit following his own distributions) into 
a given number of bins (i.e. a given set of proxy servers 
each of which has its own utility function, as will be further 
detailed in Section II). 

In our proposed framework as detailed in Section II, meta 
data information refers to the patterns about a sequence of 
itemsffor example the user’s favorable websites) infer-able 
based upon a sequence of bins (e.g. proxy sites) observed by 
an eavesdropper. This assumption is an expansion of [1] and 
[2], However, it should be noted that due to usage of proxy 
sites, an eavesdropper cannot directly observe the original 
input items, but rather bin indexes. Under our proposed novel 
framework, we introduce multi-submodularity and submodu¬ 
larity as two means of reducing the complexity level of such 
problems, namely, dividing M random items into N bins, 
under an upper-bound on leaked meta information. 

This paper represents an expansion of our previous works 
in [3], [4] where in [3] a utility function was introduced in the 
form of the average stopping time for detection of an active 
subgraph using certain queries while in [4], we developed a 
concept of information leakage through a vast set of possible 
queries. In this paper, we shift our attention from detection 
of an active subgraph to seeking tradeoff between optimizing 
utility of partitioning a set of random items and restraining 
information leakage. 

The concept of privacy has already been explored in many 
works such as [5], [6] where a general but non-mathematical 
explanation was offered. However; in our work, we go into 
further details as to what privacy represents in our framework 
and how it could be formulated into many settings. Later in 
this paper, we find it necessary to utilize the concept of multi- 
submodular set function problems and their solutions. This 
concept was widely discussed in [7] where they introduced 
a series of sufficient conditions on multi-submodular set 
functions by which the multi-submodular problem could be 
transformed into a submodular set function problem. Then, 



further discussions about the existence of a solution to the 
new problem were made. By doing so -and if a solution was 
proven to exist-, the complexity of the problem could be 
shown to be reduced from NP to polynomial. However, [7] 
did not offer any algorithmic solutions in such cases. 

The novel contributions of this paper are as follows: (1) 
we introduce a new multi-agent multi-variant optimization 
problem with a privacy leakage constraint offering specific 
accommodations for online browsing which turns out to be 
A'f'-complicatcd; (2) we introduce the novel idea concerning 
dual nature of the multi-agent optimization problems and 
their corresponding implications in such problems; (3) we 
utilize multi-submodularity property to prove the existence of 
a transformation to submodular set problems given a series 
of sufficient conditions and then concluded the existence of a 
polynomial solution; by doing so, we simplify the complexity 
of the problem from NP to polynomial (4) we offer a 
new less restrictive set of sufficient conditions as well as a 
submodular set function optimization solution algorithm with 
its corresponding calculation complexities for the specific 
case of TV = 2. 

The rest of the paper is organized as follows. In Section 
II we formulate the problem in terms of privacy and utility 
functions. We dissect what the goal and the constraints are. In 
Section III, we first introduce a revised version of the utility 
function and then find the sufficient conditions under which 
this function is equipped with multi-submodular property. 
Due to general limited knowledge about multi-submodular 
solutions and how they are developed, we then assume a 
specific case N = 2. In Section IV an algorithm for this 
specific case is developed which has a polynomial cost and 
an accuracy of 0.432. Finally, in Section V, we conclude the 
paper by reiterating what we have accomplished in this paper. 

II. System Model 

First, we propose an abstract framework to formalize the 
goal of seeking partition of N items into M bins. More 
specifically, we aim to allocate each one of 1 < i < M 
possible items (queries) to one of N output bins. There 
could be at most N M such partitions. It follows that any 
set allocation Ai, 1 < l < N M results in N sets C 
{1, 2,..., M},j = 1, 2,..., N. Each such set is defined as 

= m,j = i} 

wheTe = {o (1) 

We further assume D S^ = 0, j ^ k. Furthermore we 
have UjLi = {1,2,..., M}. Finally the size of each such 
set Sj 1 ' 1 is defined as l}-\ 

A. Probabilistic Model 

We assume at any time slot one and only one of the inputs 
is chosen with a certain probability. Thus if we use variable 
A' £ {1,2, as a representation of set of items, we 


could have P(X = i) = P("/i = 1) = 7 Tj, 1 < i < M 

as a representation of the probability of choosing item i 
from the set X where 7 $ £ {0,1}. It further follows that 
li = 1, stipulating that one and only one of M items 
is selected. These M items could represent a set of M web 
pages to be visited by a user at a particular time instant. The 
prior probability distribution of M items reflects the user’s 
favoritism toward these web pages. . 

Next, we introduce an observable random variable 
Y £ {1,2, ••• , TV}, denoting the index of the bin (the 

proxy site) employed to carry one of the M > N items. 

It follows that the probability of each bin’s appearance 
given a set allocation scheme such as Ai will be equal to 

P(Y = j\Ai) = P(Y = j\A u X = i)P(X = i\A t ) = 

Ef=i P{Y = j\Ai, X = i)P(X = i) where we have 
dropped the second conditional probability due to 
the independence between X and Ai. Furthermore 
P(Y = j\Ai, X = 1 ) = of) £ {0,, 1}. It thus follows that 

M 

P(Y=j\A l ) = J2 e ij P (X = i)^ 

2—1 

P{Y = j\A l )= Y. = ( 2 ) 

B. Revealed Information 

By choosing to allocate M items to N bins where N < M, 
we have injected ambiguity and uncertainty into the output 
binning index sequence about the input item sequence over 
a successive n visits or channel uses. In other words, if we 
originally chose to transmit n of such items, our total set of 
possible sequences would be of form X 1 ^ = [X- t X 2 ...X rl ] out 
of M n possible outcomes. From an observer’s perspective 
which can only have access to which one of N bins is 
deployed in each time slot, sequences in the form of Y n 
= [YiY 2 ...Y n ] has cardinality of at most N n < M n . Despite 
the amount of uncertainty added due to the many-to-one 
mapping between items and bins, the output sequence sill 
reveals certain amount of information regarding the patterns 
of sequences of M random items. 

This observation could be further studied by indicating 
how our allocation system resembles a coding framework 
where we have an equivalent channel whose input variable 
is X and output Y, as show in Figure 1. 



Figure 1. Coding Channel Representation of the Problem 

Under such a framework, the equivalent channel output 
sequence Y can help an eavesdropper classify the input 




sequence X" into a number of differential classes. As a 
result, information about the specific input item patterns 
is leaked to certain degree and can be measured using 
conditional mutual information I(X;Y\Ai) between X and 
Y, given a particular channel mapping (i.e. partition Ai 
relationship as illustrated in Figure 1. 

Such conditional mutual information thus measures the 
maximum number of bits of meta information about item 
sequence per channel use. Therefore, we can have at most 
2 n i(X;Y\A l ) se q Uences X" distinguishable by inferring based 
on Y n . We thus adopt I(X\Y\A{) as the privacy metric 
conditioned on a particular partition mapping A[. 

It follows that due to the combinatorial nature of a set 
allocation problem there are a total of N M possible methods 
to allocate these M items to the N sets. We can formulate 
the mutual information over a set allocation Ai , 1 <1<N m 
as: 


I{x ; Y\Ai) = H(X\Ai) - H(X\Y,, At) = 
H{Y\At) - H(Y\X,At) = H(Y\Ai) = 


where we have used the notion of H{a\, 02 , a m ) = 
— a v l°g a v and the fact that H(Y\X, Ai) = 0 because 

if both the input X and the channel scheme Ai are known, 
then output Y could simply be calculated. 


C. Utility Function 

Note that the main reason why we chose bin allocation was 
to reach a higher utility function. In this section we define 
a utility function to apply to the problem. If an allocation 
scheme Ai has resulted in s[ l \ S^ l \ ■■■, S$, we assume each 
of the N bins offer a utility function of their own based upon 
the set they have been bestowed. Every bin j thus offers a 
utility represented by It is important to note that fj 

represents a set function meaning it would change as different 
subsets of the universal set are chosen. An example for such 
a function would be if fj(Sj l ’ > ) = fj{\S^\) meaning the 
function changes as the number of members within the set 
S ( - 1 ' 1 changes. 

It then follows that the average utility function will be 
in the form of U t = Y,j=i fj( S j l) ) p (X = j\ A i) = 

Ef=r otffiiSf). 

D. Problem Definition 


where A > 0 represents a variable connecting the utility 
and the constraint to each other so as to allow comparison 
between them. We can further express this equation by 
opening it as: 

N 

E[ Q ?-M^ {,) ) + Aa ? )lo g(a?)] (5) 

j=i 

We can now see that if we define fj(Sj 1 ^) + 

Xttj log {ctp). Equation (4) is simply a sum of functions 
defined over a series of sets. We refer to these as multi¬ 
variate set functions seeing as how their values are based 
upon specific sets and variables introduced in each of these 
sets. 

III. Multi-Submodular Set Functions as a Means 
of Solution 

As mentioned previously, the problem formulated in Eq. 
(4) is NP-complicated (it is solved when a search over N M 
possible set allocations is carried out and the best allocation is 
chosen). Still, we could opt to utilize the definition of multi- 
submodular set functions so as to reduce the complexity to 
that of polynomial at the cost of accuracy. In order to do so, 
we raise concerns about possible solutions. Next, we offer 
insight as to how we could deal with each case. 

A. Imposing Multi-submodularity 

In [7], it was shown that if we can prove multi¬ 
submodularity for functions such as those formulated in 
Eq. (4), then they could be modeled as simpler problems 
(submodular set functions). We thus, aim to find the sufficient 
conditions for such occurrence. To do so, we first offer a 
review of multi-submodularity. 

As mentioned in [7], if we define M = {1,2,..., M}, 
then a multivariate function F : (2“)^ -a- R+ is multi- 
submodular if for all pairs of tuples (,S'-|..... Sn) and 
(Ti, ...Tat) G (2“)^ we will have: 

F(Si ,.., S N ) + F(Ti ,..., T n ) > F(S 1 U Ti, ..., S N U T N ) 
+F(S 1 nT 1 ,...,S N nT N ) ( 6 ) 

Since in our formulation functions are separately defined on 
different sets, the condition in Eq. (6) is simplified to the 
sufficient condition of submodularity of F^ for all sets Sj 
for Equation (4). We now need to find the sufficient condition 
for submodularity of F- when defined over a set Sj. 


Based on the previous observations we set a goal to find 
the set allocation Ai over which (a) Ui is maximized and (b) 
I(X;Y\A[) < I t h where 7 t /, represents the maximal allowed 
revealed information. 

We aim to gather both the utility and the constraint 
imposed in the form of one function we hope to maximize. 
Thus, a new maximization problem could be developed: 

max Ui + X(I th - H(a^\a^\...,a^)) (4) 


B. Imposing Separate Submodularities 

For an easier mathematical representation of the following 
derivation we denote F(Sj) = F^ . Furthermore, we denote 
;,(Sf) = /(sf). Both these denotations allude to the fact 
that once a set allocation A; is chosen, its index could be 
dropped. 

In the next step, we opt to use diminishing return property 
as the means of making certain each of these functions are 
submodular. Following is a definition of diminishing returns 



for submodular functions, after which we derive the sufficient 
conditions for the case discussed in Eq. (4). 

Diminishing Property Return dictates that if we define 
S as the universal set, a set function F : 2 s —► M + is 
submodular if, for all A, B C § with A C B and for each 
x € S — B we have [8]: 

F(Au{x})-F(A)>F{BU{x})-F(B) (7) 

Now we attempt to expand Eq. (7) for each F(Sj). However, 
to properly do so, we first need to account for the behavior 
of this function. 

We have defined F(Sj) = ajf(Sj) + A ay log (ay) where 
it seems that the function has a singular relationship with set 
Sj. However; there is a secondary relationship the function 
shares with the set ,Sj c = S — Sj where S = § represents 
the universal set. This relationship could be modeled as 
F(Sf) = (l-0j)f(S-Sf) + X(l-^) log (1-/3,) where 
we have used the fact that (3j = 1 — ay seeing as how we 
define 

pf = E ^ (») 

Thus, for any set Sj, we must find the sufficient con¬ 
ditions for the existence of diminishing property for both 
functions Fi(Sj) = ajf(Sj) + Aaylog(ay) and ^(Sy) = 
(1 — ctj)f(S — Sj) + A(1 — cij) log (1 — a,). To do so, we 
will evaluate their necessary conditions and then find their 
intersection as the final conditions (assuming they do not 
negate one another). 

Note: For any further references, we first need to address 
a series of variable and function definitions which are going 
to play a vital role in the rest of this paper: 

Definitions 

1. Any variable represented with a capital Letter represents 
a set. 

2. Any variable represented with a small letter represents 
an element. 

3. A — B represents a set containing all elements of set 
A which do not appear in set B. 

4. a x represents the probability of item x and cxa repre¬ 
sents the sum of probabilities of items mapped into a 
set A. 

5. clba represents the difference in the sum of probabili¬ 
ties of items mapped into the sets B and A which could 
be further shown as ckba — as — ola- 

6. g(C,D ) represents the 1 st order difference of a set 
function f(C) from f(C — D) where D C C which 
could be formulated as g(C) = f(C) — f(C — D). 

7. q(C, Ci, D, Di) represents the 2 nd order difference of 
a set function /(C) where Ci C C and D i C D which 
could be formulated as q(C, C\, D, Di) = g(C, D) — 
g{Ci,Di). 

8. We assume the probability of items is sorted in a 
decreasing manner such as it\ > 7T2 > ... > ttm- 


Theorem III.l. The set functions F\ (Sj) and F- 2 (Sj) and 
as a result F{Sj) are submodular if 

(1) 9 {Sj,S w ) < 0 


(2) q(Sj,S u ,S w ,S y )< 0 

(3) ^(S'y,5 W )| > Alog{~-) 

for all possible sets S w C Sj C S and S u C Sj and 
S y C S w where S is the universal set. 

The proof for this lemma is presented in the Appendix 
under Theorem III-1. In the proof, a series of sufficient 
conditions for either Fi(Sj) and /^(Sy) are evaluated sep¬ 
arately. This is done because although their conditions turn 
out to be the same, their derivations are vastly different as 
require separate discussions. It then follows that since both 
functions require the same set of sufficient conditions, the 
function F(Sj) which represents either of them being chosen, 
also follows the same set of sufficient conditions. In the 
proof, we rewrite inequality (7) for set function fy (Sj), start 
factorizing a a , a ba separately and a x and 1 together and 
impose sufficient conditions so that each of their coefficients 
is always positive. 

Unfortunately, [7] does not provide us with an algorithm 
to remodel our multi-submodular problem in a submodular 
problem, they simply prove that this could be done. Thus, in 
order to expand upon the idea of polynomial complexity of 
solution algorithms we opt to assume N = 2 and offer the 
reader the algorithm to deal with such a specific case. We 
then calculate the complexity imposed by the algorithm to 
further stress the benefits of using such an idea in spite of 
accepting error. 

C. Specific Case of N = 2 

As mentioned previously, in order to show the applicability 
of submodular functions we choose to reiterate the utility 
function dictated in Eq. (4) for when N = 2: 

i max M a { ^f(S[ l) ) + (1 - a[ l] )f{S - S[ l) ) 

+A {Ith + log (a^ } ) + (1 - a^ } ) log (1 - a[ l) )) = T(5i) 

(9) 

As can be seen, the problem is still exponentially complex 
seeing as how we need to search over 2 M possible solutions 
to find the optimal. Thus, once again we aim to impose multi¬ 
submodularity (in this case simplified to submodularity) on 
the new utility function. For the utility function above the 
same results derived for a general N could be used as a 
set of sufficient conditions. However, taking into account the 
joint relationship between the 2 sets and writing the same 
Inequality (7) for Equality (9) we are able to find a less 
restrictive set of sufficient conditions for the submodularity 
of this utility function as indicated below: 

Lemma III.2. When N = 2, the function in Eq. (9) is 
submodular if 

(1) g(s[ l \stt)< 0 

(2) 2\g{S?\S$)\ > A log {K),K < (^) 2 

(3) q(Sy,sV\sti\sb l) ) < 0 

for all possible sets S$ C S^ C S and Su ^ C S^ and 
Sy ^ C S^ where S is the universal set. 




The proof for this lemma is presented in Appendix under 
Lemma III-2. In the proof, we rewrite inequality (7) for 
set function described in Eq. (9), start factorizing a a, ctB A 
separately and a x and 1 together and impose sufficient 
conditions so that each of their coefficients is always positive. 
In the following section, we will offer a method of solving 
a problem as introduced in Eq. (9). 

IV. SUBMODULAR SOLUTION 

Starting by [9] there has been monumental work done 
over greedy algorithms with constraints (as long as they 
introduce down-monotone solvable polytopes) with a solution 
proximity of 1. Later [10] introduced a solution proximity 
of 0.372. Finally [11] proved that this proximity could be 
increased to 0.432 in maximization problems which is quite 
close to the no-constraint solution of a symmetric problem. 

It is important to note that while all these papers dealt 
with the issue of constraints, they assumed much more 
complex constraints than we are dealing with in this paper. 
Our only constraint is that a[ r> < a s where we assume 
h{a s ) = Ith, ct s < 0.5 which is obviously a down-monotone 
constraint. Thus, we can simply use the results from their 
work to create our own algorithm to find the submodular 
function solution to our problem. We present: 

Algorithm 1: Submodular Function Solution to the problem as 

described in Eq.(4) 

1. Let Si = argmax eex= { 1 m jT[S i = {e}] while 
|0.5-a^’ 1) | > |0.5 — a s |. 

2. If there is an element e £ X \ Si such that 
T[Si + {e}] > T[Si] and |0.5-a[ !) -a e | > |0.5-a s |, 
let Si = Si + {e}. 

3. If there is an element e £ Si such that T[Si \ {e}] > 

T[Si] and |0.5 - + a e \ > |0.5 - a s |, let Si = 

Si — {e}. Go to Step 2. 

4. Return maximum of T[Si] and T[X \ Si]. 


where we know that at the very last step T[Si] = T[X\Si]. 
Now we opt to calculate the complexities of this method. 
Steps 2 and 3 could repeat (M — 1) + (M — 2) + ... + 1 = 
M(M+1) t j mes eac j 1 w j 1 jj e ever y item could be removed and 
thus replaced a total of 2 M times. Thus the total complexity 
of steps 2 and 3 is equal to M 2 (M + 1) = 0(M 3 ). The 
complexity of step 1 is also equal to M. Thus the total 
complexity of the solution is equal to 0(M 3 ). 

This polynomial solution simply makes certain the max¬ 
imal function obtained is at least 0.432 times the optimal 
objective function. This range of error occurs because in 
this method, we are removing and adding members from 
and to the set ,S’i one by one. Thus, at each decision point 
we are making one locally optimal decision. However, it is 
widely known that a locally greedy method is not necessarily 
globally optimal [12], 

V. Conclusions 

In this paper, we introduced and formulated a problem 
widely regarded in online browses. To do so, we revisited the 


concept of privacy leakage discussed in both other and our 
own previous publications. We further introduced a utility 
function based upon the user’s utilization of the network. 
We showcased how the problem formulation results in a 
multi-agent multi-variate problem which is VC-complicated. 
We then introduced the concept of submodularity and multi¬ 
submodularity which help reduce the complexity of such 
problems to that of a polynomial at the cost of some accuracy. 
We derived a series of sufficient conditions which would 
guarantee the existence of a solution. To do so, we introduced 
a novel observation of the behavior of such multi-agent 
multi-variate set functions which we called duality. Once the 
existence of such solutions was guaranteed, we introduced 
algorithms that could help us when N = 2 (but the original 
complexity is still combinatorial) and showcased how they 
help reduce the complexities. 
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Appendix 


A. Theorem 111.1 

Proof. For an easier understanding, our proof of the theorem 
is broken into two sections: 

Sufficient Conditions for Fi(Sj): 




Starting for Fi(Sj), by rewriting Eq. (7) we will have: 

(a A + a x )f(A + {x}) + \{a A + a x ) log ( a A + a x ) 
~a A f(A) - Xa A log (a A ) > 

(a A + a BA + a x )f(B + {x}) 

+\(a A + a BA + a x ) log ( a A + a BA + a x ) 

~{a A + a BA )f(B) - \{a A + a BA ) log (a A + a BA ) 

We can then factorize the inequality as 

0 tBA[-f{B + {a:}) + f{B) - X log (aA + a BA + a x ) 

+A log (■ a A + a BA )j + a x If (A + {x}) 

—f(B + {x}) + A log {a A + a x )-X log (a A + a BA + a x )] 
+a A [f(A + {x}) - f(B + {x}) - f(A) + f{B) 

+X log (a A + a x ) — X log (a A + a BA + a x ) 

-X log (a a) + X log {a A + aBA)} > 0 

Using cj and q set functions, we can then rewrite the factor¬ 
ization as: 

a BA [-g(B + {x}, B) + A log [——^-]] 

1 + c A+aBA 

+a x [-g(B + {x}, A + {x}) + Alog ^ \ BA ]] 

OlA+Oi x 
^ _|_ &BA 

+a A [-q(B + {x},B,A +{x},A) + Alog > 0 

' OtA+Ota: 

Since the above inequality needs to hold true for all possible 
sets of A C B, x B, we aim to determine the maximal 
amount enforced by the above set of inequalities. The first 
factorization results in two inequalities: (1) g(C,D ) < 0 and 
(2) \g(C, D)\ > Amax(log (1 + ^)) for all sets D C C. 
To find the maximum of such a limit, we need to impose the 
one item with highest probability to {a:} and assume the one 
lowest probability item to set B. Then the above inequality 
is maximized. 

The second factorization results in two inequalities: (1) 
9 {C,D) < 0 and (2) \g(C,D)\ > Amax(log(l + 
for all sets D C C. To find the maximum of such a limit, 
we need to impose the item with lowest probability to {x} 
and that a A = 0 and then have a BA = 1 — a x = a B . 

Once again, it is important to note that by doing so, we 
are removing the possibility of a A = 0 and a x = 0 as a 
case. In other words, we are assuming that a A + a x = 0 is 
not a scenario we need to discuss. We will now demonstrate 
why such an assumption is accurate. 

When a A + a x = 0, we can rewrite inequality (7) in the 
following format: 

0 > a BA f(B) - a BA f{B) = 0 

which is always true. 

The third factorization could be simplified. The logarithm 
argument consists of a nominator greater than denominator 
thus resulting in the overall logarithm argument to be posi¬ 
tive. We thus only need to impose that q(C, C \. D 1 1) \ j < 0 
for all sets C\ C C, D\ C D and D C C. 


It then follows that the probability of each item is 
sorted in a decreasing manner as 7Ti,.... ttm, we can 
write the set of 3 sufficient conditions for submod¬ 
ularity of set function F-\ (Sj),j = 1 as (1) 

9 (C,D) < 0 (2) < 0 (3) \g{C,D)\ > 

X log[max(l + . I • 1 )] = Alog(^r) for all sets 

Ci C C, 7), C f5 A and D CC. 

Note 1: It is important to note that any maximization is 
carried out by assuming that the lower bound cannot go to oo. 
However there are times when specific set allocations could 
result in denominators being equal to 0. Such set allocations 
then need to be addressed separately by rewriting Inequality 
(7) and finding their specific sufficient conditions. 

In this proof, we have two denominators a B and a A + 
a x . We need to address each specific condition under which 
either of these are equal to 0 to find their specific sufficient 
conditions. 

First, we assume that a B = 0. This in turn means that 
B = 0. Furthermore, since /l C If we can conclude that 
a A = a B = 0 and then rewrite inequality (7) in the following 
format: 


a x f({x}) > a x f({x}) 

which is always true. 

Second, we assume that a A + a x = 0; however, this 
could never occur seeing as how a x > 0. Thus, the second 
possibility does not need to be further checked. 

Sufficient Conditions for F 2 (Sj): We now follow the 
same method for FASj)^ = 1, ...,7V by rewriting Eq. (7): 

(1 a A a x )f(S - A - {x})+ 

A(1 a A a x ) log {l-OLA- a x ) - (1 - a A )f(S - A) 

—A(1 - a A ) log (1 - a A ) > 

(1 a A a BA - a x )f{S - B - {x}) 

+A(1 - aA - a BA - a x ) log (1 - a A - a BA - a x ) 

-(1 -a A - a BA )f{S - B)- 
A(1 a a a BA ) log (1 -a A - a BA ) 


We can then factorize the inequality as 

a BA [f(S - B - {x}) - f(S - B) 

—Alog (1 + --^-)] 

1 — a a — a BA — a x 

+a x [-f(S - A - {x}) + f(S - B - {x}) 

—Alog (1 + ---)] 

1 — a a — a x — a BA 

+a A [- f(S -A- {x}) + f(S — B — {x}) + f(S - A) 

-f(S - B) + A log (-- - -)] 

1 - a A - a x 1 - a a - a BA 

+1 [f(S -A- {x}) — f(S — B — {x}) - f(S - A) 
+/(S-B) + " 


1 -OC A 
> 0 


1 — a a ~ a BA — a x 



Using g and q set functions, we can rewrite the factorization 
as: 


0tBA[-g{S -B,S-B- {x}) 

—A log (1 + ----)] 

1 — a a — &ba — o x 

+a x [-g(S -A - {x}, S - B - {x}) 

CUBA 


-A log (1 + 


-)] 


1 — a a — a x — as a ' 
+a A [q{S — A, S — A — {x}, S — B, S — B — {x}) 

, 1 — 01A 1 — OiA — OlBA — OL x , 

+Alog (-- - -)] 

1 — a A — a x 1 - ola - olba 

+l[-q(S -A,S-A- {x}, S — B, S — B — {x}) 

, 1 - a A - a x 1 - ola — CUBA >, . „ 

+A log (—----)] > 0 

1 — OtA 1 ~ 0 A ~ CUBA - Ol x 


Since the above inequality needs to hold true for all possible 
sets of A C B, x (ji B, we aim to determine the maximal 
amount enforced by the above set of inequalities. The first 
factorization results in two inequalities: (1) g(C,D ) < 0 and 
(2) \g(C, D)\ > A log (1 + ) for a11 sets D - C - 

To find the maximum of such a limit, we need to impose the 
one item with highest probability to {x} and assume that olb 
is maximal while still less than 1 — a x . This limit is imposed 
so that the denominator is not equal to 0. 

The second factorization results in two inequal¬ 
ities: (1) g(C,D) < 0 and (2) \g(C,D)\ > 

A?nax(log (1 + )) for all sets D C C. To 

find the maximum of such a limit, we once again need to 
impose the one item with highest probability to {x} and 
assume that a A = 0 and as is maximal while still less 
than 1 — a x . This limit is once again imposed so that the 
denominator is not equal to 0. 

The third and forth factorizations could be simplified. We 
could write them as 


q(S~A,S~A~ {x}, S — B, S — B — {»})[-1 + a A ] + 
1 — OLA 1 ~~ OiA — CUBA ~ Ux 


\a A log ( 
+A log ( 


l-a A -a x 
1 - ola 


1 — a A — Cuba 
1 OLA OL x 


)>0 


' 1 — a A ~ OLB A 1 — OLA — OLB A — OL x 

Depending on whether the sum of logarithmic arguments on 
the LHS is positive or negative, we can write two inequalities: 

1 — OLA 1 — OLA~ OLbA ~ OL x 


(1) Xola log ( 


+A log (-- 

1 -Ot A - 


1 - a A - OL x 1 — OLA — OLB A 
1 - OL A 1 OL A Ol x 


)>o 


OLA ~ OLB A 1 — OLA — OLB A — OL x 

q{S~A,S~A~ {x}, S — B,S — B — {x})[-l + a A ]+ 

1 — OLA 1 ~OLA — OLB A ~ OL x 


Aa A log ( 
T A log (: 


1 - OLA - OL x 
1 - OLA 


1 - OLA — OLB A 
1 OLA OL x 


-)> 


' 1 — OLA — OLB A 1 — OLA — OLB A — O x 

q{S~A,S~A~ {x}, S — B,S — B — {x})[-l + a A ] 
+Xa A log (-— 1 ——-) > 0 

1 — ola — Ob A 


where we have simply summed the two logarithmic argu¬ 
ments. It then turns out that the logarithmic argument is 
always positive seeing as how the nominator of the fraction 
inside it is greater than the denominator. It thus follows that 
we merely need to impose q(C, Ci, D, Di) < 0 for all sets 
G'i C C. D \ C I) and D C C to help above inequality hold 
true. A second scenario dictates when: 


, , , 1 - a a 1 — a a - oba - o x 

1) Xa A log (-- - - 

1 Ol A i 


1 - OA 


o x 1 — OA - oba 
1 — oa - a x 


)<0 


+Alog (-- 

1 — ola — oba 1 ~ oa — oba — o x 

q{S~A,S~A~ {x}, S — B, S — B — {*})[-1 + a A ] + 

1 — oa 1 oa oba o x 


Xa A log ( 
+Alog (- 


1 - a a - a,j 

1 -o A 


1 — oa — oba 
1 Ola o x 


■)> 


' 1 — oa — oba 1 ~ oa — oba — o x 
q(S — A,S — A — {x}, S — B,S — B — {*})[-1 + a A ] 
1 ~ o A 


+X log ( 


1 — a a — oba 


? 

)>0 


where we have once again simply summed the two loga¬ 
rithmic arguments. It again turns out that the logarithmic 
argument is always positive seeing as how the nominator 
of the fraction inside it is greater than the denominator. 
It once more thus follows that we merely need to impose 
q{C,Ci,D,D\) < 0 for all sets C\ C C, D\ C D and 
DCC to help above inequality hold true. 

Note 2: Once again, note that any maximization is carried 
out by assuming that the lower bound cannot go to oo. 
However there are times when specific set allocations could 
result in denominators being equal to 0. Such set allocations 
then need to be addressed separately by rewriting Inequality 
(7) and finding their specific sufficient conditions. 

In this proof, we have one denominators 1 — as — o x 
appearing twice. We need to address the specific condition 
under which this amount is equal to 0. This could only occur 
when B = {x\ r . In such cases, we can rewrite inequality 
(7) in the following format: 


(l-a A - o x )f(S - A - {x})+ 

A(1 -a A - a x ) log (l-o A ~ a x ) - (1 - o A )f(S - A) 

—A(1 - a A ) log (1 - a A ) > -(1 ~o A ~ o B a)/{S - B ) 
-A(l - oa- oba) log (1 - a A - o B a) 

OBA{~f{S - B) - X log (1 - a a - o B a)}+ 
o x {-f{S -A- {x}) - A log (1 -a A ~ Qba)}+ 
o A {~f(S -A- {x}) + f(S -A)- f(S - B)— 

X log (1 -OA- a x ) + A log (1 - a a) 

-A log (1- ola- oba)} + 1 {f(S -A - {x}) - f(S - A)+ 
f(S - B) + X log (1 -o A - a x ) - X log (1 - a A ) 

+X log (1 -o A - o B a)} > 0 


By adding and removing three factors of f(S — B — {x}), 
A log (1 — a a — o x ) and A log (1 — a a — oba) to each of 
the factorizations above and using the definitions of g and q 



functions as previously, we can simplify the above inequality In such a case, diminishing return property requires that 
in the following format: 


a B A{~g{S - B,S - B - {x}) + A log (1 - a A - a x )}+ 
a x {-g(S - A - {x}, S - B - {x}) + Alog(l - a A - a BA )j 
+a A {q{S — A, S—B,S — A — {x}, S-B- {x})+ 

A log (1 - a A )}+ 

1 {-q{S — A, S — B,S — A — {x}, S-B- {x})} 

—A log (1 - a A )}+ 

{A log (1 - a A - a x ) + A log (1 - a A - a BA )} 

{-a BA -a x -a A + 1} > 0 

Using the fact that a BA + a x + a A = 1 and merging the 3rd 
and 4th factorizations together, we will have: 

a BA {-g(S - B,S - B - {x}) + A log (1 - a A - a x )}+ 
u x {-g{S -A- {x}, S-B - {x})+ 

A log (1 - a A - a BA )} + (1 - a A ) 

{—q(S — A, S — B,S — A — {x}, S-B- {x}) 
-Alog(l - a A )} > 0 


(■ a A + a x )f(A + {x}) + (1 - a A - a x )f{S - A- {x}) 
+{a A + a x ) log (a a + a x ) 

+(1 - a A - a x ) log (1 - a A - a x ) - ( a A )f(A) 

-(1 ^ a A )f(S -A) - (i ol a ) log ( a A ) 

-(1 - oca) log (1 - a A ) > 

{ola + a BA + a x )f(B + {x}) 

+(1 - a A - a BA - a x )f(S - B - {x}) 

+(cx A + ol ba + a x ) log ( a A + a BA + a x ) 

+(1 - a A - a BA - a x ) log (1 - a BA - a A - a x ) 

-( a B )f(B ) - (1- a A - a BA )f(S - B) 

-(a A + a BA ) log (a A + a BA ) 

-(1 - a A ~ a BA ) log (1 -a A ~ a BA ) 


which would hold true as long as (1) g(C,D) < 0 for 
all sets C and (2) \g{C,D)\ > Amax(log Q ' j and (3) 
q(C,Ci,D,Dx) < 0 for all sets C\ C C,Di C D and 
D C C. It could be perceived that this specific set allocation 
demands a different lower bound for the absolute value of 
the first order difference of function / over sets. 

Thus 3 sufficient conditions for submodularity of 
F' 2 (Sj),j = 1 are developed: (1) g{C,D) < 0 (2) 

q(C, Ci,D, Di) < 0 (3) \g(C,D)\ > Alog(^-) for all sets 
Ci C C, D 1 C D and D C C. 

Finally, the intersection of the two sets of sufficient con¬ 
ditions for submodularity of Fi(Sj) and FASj) gives us the 
sufficient conditions for submodularity of F(Sj) which turns 
out to be the same as either of theirs. 

□ 


B. Lemma III.2 


Proof. For the specific case TV = 2: 


F(A) = a A f{A) + (1 - a A )f(S - A) + a A log (a A ) 
+(1 - a A ) log (1 - a a) 


We then factorize this inequality in the following manner: 


a BA {~f(B + {x}) + f{B) + f(S — B — {x}) - f(S - B) 


+Alog (- 


a A + a BA \ — ol A — ol ba - OL a 


)}+ 


v ol A + a BA + ol x 1 — a a — u BA 
ax{f{A + {x}) - f(B + {x}) - f(S -A- {x}) 
+f{S - B - {x})+ 
a A + a x 1 — a A — a x — a BA 


A log (- 


-)} 


' OLA + OL x + OL BA 1 — OLA ~ OL x 

+a A {f{A + {x}) - f(B + {x}) - f(S -A- {x}) 
+f(S -B- {x}) - f(A) + f(B) + f(S -A)- f(S - B) 


log( 


a a + ol x 


1 - OL A — OL x — OL B A OLA + OL BA 


OLA + OL x + OL BA 1 — OLA — CL X 
1 — Q-A 


OLA 


1 — OLA — OL BA 

+{f(S -A- {x}) — f(S — B — {x}) - f(S - A) 

1 — OLA - OL x 1 -OLA- OL BA 


+f{S — B) + X log ( 


1 — OLA — OL x — OL BA 1 — OLA 

> 0 


)} 



We then use the definitions of set functions g and q to 
simplify the above inequality: 


a B A{-g{B + {x}, B) - g(S - B,S - B - {x}) 


+Alog(- 


ola + CUBA 1 — ola — oiba - Oi a 


)}+ 


' a a + a B a + o x 1 — a a — o B a 

a x {-g(B + {x}, A + {x}) - g{S - A - {x}, S - B - {x}) 
a a + a x 1 - ola - o x - olba 


+A log (- 


-)} 


OLA + O x + OLBA 1 ~ OLA — O x 

+a A {~q{B + {x}, B,A + {x}, A) 

+q(S — A, S — A — {x}, S — B, S — B — {x}) + A 

OLA + OL x 1 — OLA — Ol x — as A OLA + OLBA 


log( 


OLA + OL x + OLBA 1 — OLA — 0l x 
1 - OLA 


OLA 


1 OLA OLBA ‘ 

+{-q(S — A, S — A — {x}, S — B, S — B — {x}) 

1 — OLA ~ OL x 1 — OLA ~ OLBA s 


+A log ( 


1 — a A — 0L X — OLBA 1 — OLA 

> 0 


L )} 


We aim to find sufficient conditions so that the above inequal¬ 
ity can hold true. The first factorization is the coefficients of 
olba > 0. We thus need to ascertain that the coefficient is 
also positive. To do so, we impose that 


-2 g(C,D) + A log ( 


Ola + OLBA 1 — OLA — Olba — OLa 


OLA + OLBA + Ol x 

> 0 


1 — OLA — OLBA 


)} 


for all possible sets D C C. We thus need to impose two 
sufficient conditions: 


The first fraction is maximized when a a 0, a x ~ tt m and 
a B A = 1 — 7 tm while the second fraction can never be as 
high. It thus follows that: 

K < X log (—— ) 2 
7TM 

The 3 rd and 4 th factorizations could be merged together 
in the following manner: 


[-1 + a A ]q(S -A,S-A- {x}, S — B, S — B — {x}) 
— aAq(B + {x}, B : A 4~ {a:}, A') T A 
ola+ a x 1 ola — ol x - olba ola + a B a 


{a A log ( 


a a + ol x + olba 1 ~ ola ~ ol x 
1 - OLA x 


OLA 


1 OLA OLBA 


log(: 


1 OLA OL x 


1 — a a — olba 


1 -OLA 


' 1 — a a — ol x — olba 

We can show that: 

1 — ola — olba 
olba 1 — OLA 


)}>0 ( 11 ) 


i ( 1 OLA OL x 

log(^- 

f — a a — cx x — * 


OLA - OL x 

\ _ a BA 

tog q o R “ 4 )>0 

1 -ctA — a x 

We can thus find a lower bound for the LHS of the inequality 

( 11 ): 


LHS > [—1 + aA]q{S — A, S — A — {x}, S — B, S — B — 
-a A q{B + {x}, B, A + {x}, A) 
a a + ol x a a + olba , 


+a^Alog(- 


ola 


OL x 


■ OLBA 


OLA 


-)>0 


max(|A log ( 


(1) g(C,D)<0yC 

(2) 2\g(C,D)\> 

a A + OLBA 1 — OLA — OLBA — OL x 
OLA + OLBA + OL x 1 — OLA — OLBA 


)}|) 


The second factorization is the coefficients of a x > 0. 
We once again need to ascertain that the coefficient is also 
positive. To do so, we impose that 


?nax(|A log (- 


(1) g(C, D) < 0, VC 
(2) 2\g(C, D)\ > 

CLA + 0L X 1 - OLA - OL x - as A 


)l) 


ola + ol x + olba 1 ~ ola ~ ol x 

for all sets D C C. We could see that the RHS of the 
secondary condition could be written as: 

OLBA w, , OLBA 


Amax(log ((1 + 


ola 


-)(1 


1 — a a — ct 


-))) = K 


( 10 ) 


Furthermore, we can show that 

OLA + OL x OLA + OB A 


log(- 


OA 


ob a 


OA 


) = log ( 


Ot-BA 

OL A 


1 \ CX-BA 


)>0 


Thus, all we need to impose to guarantee the 3 rd 
and 4 th factorizations do not cause any ambiguities, is 
q(C, Ci, -D, Di) < 0 for all sets Oi C C,fii C and 
DCC. 

Note 3:: In this case there are two lower bound de¬ 
nominators a a + Ob a + a x and 1 — ola — oba- The 
first denominator cannot be equal to 0 because a x > 0. 
If the second denominator is equal to 0, that means that 
a a = 1 — a x < alphas < 1 — o x which means that A = B 
thus inequality (7) will definitely hold true. 

□ 


It is clear to see that the two fractions could not be maximal 
at the same time seeing as how (1) the maximal occurs 
when denominator of each is close to zero and (2) their 
denominators have opposing behavior. It thus turns out that 
the RHS as indicated in Eq. (10) is limited by: 

I\ < A log (max(l -|- —— —)max(l H-——-)) 

a A + a x 1 o A ol x ' 


M) 



